Augmented reality extended view

ABSTRACT

Disclosed embodiments include systems, devices, and methods for extending a view of an interface by way of augmented reality glasses. Web page content can be identified for presentation outside the purview of a web browser. The content can be saved to a network-accessible store. Subsequently, a request is sent, including the location of the network-accessible store, to augmented reality glasses. The content can be acquired from the location by the augmented reality glasses. Subsequently, the content can be rendered by the augmented reality glasses in a manner that extends the view beyond the web browser and physical display.

BACKGROUND

A web browser is a program that can allow a user to explore the WorldWide Web or simply the web. For example, a user can employ a web browserto request a web page from a website. In response, the web browseracquires content from a web server and presents the content within a webbrowser window, which is rendered on a display of a computing device.Web browsers can provide several additional features, such as tabs thatallow multiple web pages to be open simultaneously. Tabs enablenavigation between currently displayed and hidden web pages. Selectionof a tab activates display of a web page associated with that tab.

SUMMARY

The following presents a simplified summary to provide a basicunderstanding of some aspects of the disclosed subject matter. Thissummary is not an extensive overview. It is not intended to identifynecessary elements or to delineate the scope of the claimed subjectmatter. Rather, this summary presents some concepts in a simplified formas a prelude to the more detailed description presented later.

Briefly described, disclosed embodiments may relate to an augmentedreality extended view. A web browser can include functionality thatextends the display of web content beyond a browser window and theconfines of a physical display device. The web browser can requestcontent presentation by an augmented reality device, such as augmentedreality glasses, through wireless communication or code display.Further, the web browser can provide content for display by theaugmented reality device at a network-accessible location communicatedwith the request. Additional information specifying the position of thecontent can also be provided at the network-accessible location, amongother things. The augmented reality device can acquire the content andposition information from the location and project the content at theposition specified within an extended view overlaid and centered on thebrowser. The augmented reality extended view provides an immersiveexperience in which content is superimposed onto the real-worldenvironment and outside the purview of a web browser and display device.In one instance, the content presented in the extended view can becopied and subsequently pasted within a web browser window. Further,content presentation in the extended view can be discontinued inresponse to events such as closing a browser window or tab or a userrequest, among other things.

According to one aspect, disclosed embodiments may include a system thatcomprises a processor coupled to a memory that includes instructionsthat, when executed by the processor, cause the processor to identifycontent associated with a webpage being displayed on a web browser forpresentation outside a purview of the web browser, save the content tonetwork-accessible storage, and generate a request, including a locationof the network-accessible storage, to an augmented reality device topresent the content in an extended view overlaid on the web browser. Theinstructions can further cause the processor to identify contentassociated with a web browser extension that generates a virtual paymentcard or identifies a deal on a product or service during onlineshopping. In one instance, the instructions can further cause theprocessor to generate a visible code and display the visible code on awebpage of the web browser to send the request. The instructions cancause the processor to initiate wireless communication with theaugmented reality device in another instance. Further, the instructionscan cause the processor to set a flag to continue presentation ofcontent in the extended view or discontinue the presentation of thecontext in the extended view. The instructions can also cause theprocessor to detect the closing of a web browser and set a flag todiscontinue the presentation of the content in the extended view.Furthermore, the instructions can cause the processor to acquire datacopied from the extended view and save the data on a clipboard to enablethe data to be pasted on a web page in the web browser. In one instance,the instructions can further cause the processor to save the displayposition of the content in the extended view at the location.

According to another aspect, disclosed embodiments may include a methodcomprising identifying content associated with a webpage being displayedon a web browser for presentation outside a purview of a web browser,saving the content to a network-accessible data store, and generating arequest, including a location of the network-accessible data store, toan augmented reality device to present the content in an extended viewoverlaid on the web browser. The method can further comprise identifyingnon-displayed web page content for presentation outside the purview ofthe web browser. The method can also comprise identifying web pagecontent generated by a web browser extension that generates a virtualpayment card or identifying web page content generated by an onlineshopping browser extension. Further, the method can comprise setting aflag to discontinue presenting the content in the extended view. Themethod can also comprise generating and displaying a visible code on aweb page of the web browser to send the request.

Per another aspect, disclosed embodiments may include a methodcomprising identifying a request associated with content being displayedon a web browser, accessing a network-accessible data store specifiedwith the request, acquiring content from the network-accessible datastore, and rendering the content in an augmented reality view thatextends beyond physical boundaries of a display device that displays theweb browser. The method further comprises acquiring a position from thenetwork-accessible data store and rendering the content at the positionin the augmented reality view. Further, the method comprises copying atleast a portion of the content and transmitting the at least a portionof the content to a computing device for pasting within the web browser.The method also comprises checking the flag specified on thenetwork-accessible data store and discontinuing the rendering when thecontent when the flag is set.

To accomplish the foregoing and related ends, certain illustrativeaspects of the claimed subject matter are described herein in connectionwith the following description and the annexed drawings. These aspectsindicate various ways in which the subject matter may be practiced, allof which are intended to be within the scope of the disclosed subjectmatter. Other advantages and novel features may become apparent from thefollowing detailed description when considered in conjunction with thedrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an overview of an example implementation.

FIG. 2 is a block diagram of an example view extension system.

FIG. 3 is a block diagram of an example browser extension.

FIG. 4 is a block diagram of an example augmented reality deviceextension.

FIG. 5 is a flow chart diagram of a method of presenting an extendedview

FIG. 6 is a flow chart diagram of an augmented reality presentationmethod.

FIG. 7 is a flow chart diagram of a method of content copying from anextended view.

FIG. 8 is a block diagram illustrating a suitable operating environmentfor aspects of the subject disclosure.

DETAILED DESCRIPTION

Displays of web content are typically confined to a web browser window,limited by the constraints of the browser's graphical user interface, aswell as the physical display device (e.g., a monitor) on which thebrowser is displayed. For example, content can be acquired from a webserver and displayed by a browser rendering engine in a browser window.In many instances, all content is not wholly visible within the browserwindow as the content size can exceed the presentation space.Consequently, browsers provide horizontal and vertical scrolling toaccess all content within the window. Further, web browsers haveintroduced tabs to allow multiple documents or panels to be containedwithin a single window. However, there can be only one visible, activepanel, and scrolling may still be required to view all content in thepanel. Moreover, the physical space allotted for displaying the webbrowser can be limited by the hardware associated with the computingdevice. For example, a portable device, such as a small laptop orsmartphone, may have a smaller display than a desktop computer display.

In addition to conventional web content, web browser extensions canprovide additional content for display. For example, an online shoppingextension can search for promotional codes, deals, and the lowest priceon a product or service. In another instance, an extension can generatevirtual payment cards to purchase products. Web browser content can bere-rendered to include extension content. The extension content can beoverlaid on original browser content in a window, or a new tab thatincludes the extension content can be opened.

It can be difficult and inconvenient to access particular content withinthe purview of a web browser. Accessing content of interest can entailactivating particular tabs and scrolling through content. In someinstances, content can be distributed across tabs causing a user tocycle through activation and deactivation of tabs. Further yet, the viewof content within a web browser window is constrained by the physicaldimensions of a display associated with a computing device. For example,a small display may necessitate more scrolling, as well as zooming inand out, to access content than a larger display.

Disclosed embodiments may provide an extended augmented reality view ofweb-based content. A web browser can request that content be presentedon augmented reality glasses. The request can include content as well asposition data. The content can include a web page, image, video,animation, code, or a combination. In one instance, the request,content, and position data can be wirelessly communicated directly topaired augmented reality glasses. Alternatively, the request can includea network address, and the address can be accessed to acquire thecontent and position data, among other things. After content andposition data are acquired, the augmented reality glasses can projectthe content at the position specified within an extended view overlaidand centered on the browser. In one instance, the content presented inthe extended view can be copied and transmitted to the browser, forexample, to fill in payment fields. After a specified time or inresponse to the closing of a browser or user request, the augmentedreality glasses can terminate the display of the content.

Various aspects of the subject disclosure are now described in moredetail with reference to the annexed drawings, wherein like numeralsgenerally refer to like or corresponding elements throughout. It shouldbe understood, however, that the drawings and detailed descriptionrelating thereto are not intended to limit the claimed subject matter tothe particular form disclosed. Instead, the intention is to cover allmodifications, equivalents, and alternatives falling within the spiritand scope of the claimed subject matter.

Referring initially to FIG. 1 , a high-level overview of an exampleimplementation 100 is depicted. The implementation 100 may include acomputing device 110, display 112, web browser 120, a view browserextension 130, augmented reality (AR) glasses 140, extended view 150,network 160, and servers 170.

The computing device 110 may include any processor-based device with adisplay 112, as described in further detail later. As shown, thecomputing device 110 is a laptop computer. However, the computing device110 could also be a desktop computer, tablet, or smartphone, among otherthings. Further, the computing device can support wirelesscommunication, including a wireless personal area network (WPAN),wireless local area networks (WLAN), wireless wide area network (WWAN),and cellular network or mobile network.

As shown in FIG. 1 , the web browser 120 is currently being presented onthe display 112. The web browser 120 may be an application executed onthe computing device 110 for retrieving, presenting, and traversing aninformation resource on the web. An information resource can be a webpage, image, video, or other content identified by a uniform resourcelocator (URL) or web address. Furthermore, the web browser 120 cansupport browser extensions for customizing a web browser. Extensions canmodify a user interface, block ads, manage cookies, as well as providecustom scripting for web pages. For example, an online shoppingextension can provide promotional codes and identify the lowest pricefor a product or service. As another example, an extension can supportthe generation and use of virtual payment cards.

The view browser extension 130 is a browser extension that enables anextended view 150 beyond the purview of the web browser and physicaldimensions of the display 112. In one instance, the view browserextension 130 can operate with respect to other browser extensions or beintegrated within the browser extensions themselves. However, the viewbrowser extension 130 can operate with respect to conventional webinformation resources, such as non-customized web pages.

The view browser extension 130 can identify information resources forpresentation in the extended view 150. The resources can be stored at alocation on a network-accessible server. Additionally, the view browserextension 130 can include position information regarding where in theextended view 150 the information resource should be located. Otherinformation such as the time the resource should be presented or removedcan also be included.

Further, the view browser extension 130 can request the resources bepresented by the augmented reality glasses 140. In one instance, therequest, location of one or more resources, and other information can becommunicated wirelessly from the computing device 110 to the augmentedreality glasses 140. For example, Bluetooth communication can beemployed between the computing device 110 and the augmented realityglasses 140. The web browser can display a code 122 the augmentedreality glasses 140 can read in another embodiment. Here, the code 120is a quick response (QR) code or two-dimensional barcode. The QR codecan include the location of the resource to display, alone or incombination with other information or executable code.

Disclosed embodiments may include an augmented reality device, such asthe augmented reality glasses 140. Augmented reality glasses 140 maysometimes be referred to as smart glasses or an augmented realityheadset and may include head-wearable lenses that enhance reality byprojecting content onto the lenses for a more immersive experience.Augmented reality glasses 140 may superimpose content onto thereal-world environment, such as images, video, and text. The augmentedreality glasses 140 are operable to acquire information resources andproject the resources into the extended view 150, which may correspondto an area outside the purview of the web browser 120 and the physicaldimensions of the display 112.

The augmented reality glasses 140 can receive a request for display anda network-accessible location where the information resources or contentare stored. In one embodiment, the request and network-accessiblelocation can be transmitted and acquired through wireless communication.For instance, short-range wireless communication systems such asBluetooth Wi-Fi, ZigBee, Ultra-wideband (UWB), and infrared (IR) can beutilized by the computing device 110 and the augmented reality glasses140. In an alternate embodiment, the augmented reality glasses caninclude a camera that can detect and read the code 122 displayed by theweb browser 120 or a web page presented by the web browser 120. The code122 can be a mechanism for representing a network address in a visualand machine-readable form. For example, the code 122 can correspond to aone-dimensional or two-dimensional barcode. After the location oraddress is acquired, the augmented reality glasses 140 can acquireinformation resources or content from a network-accessible server andpresent such content in the extended virtual reality view overlaid onthe web browser. In one instance, the augmented reality glasses can alsobe extended to support the extended view 150. For example, anapplication, applet, plugin, extension, or the like can be acquired andinstalled on augmented reality glasses.

The network 160 can correspond to a local area network (LAN) or widearea network (WAN), among others. In accordance with an embodiment, thenetwork 160 can correspond to the Internet or a global system ofinterconnected computer networks and devices that uses the Internetprotocol (TCP/IP) to communicate. The Internet can support inter-linkedhypertext documents and services on the web, among other things.

The server(s) 170 correspond to hardware, software, or both that providefunctionality or services to multiple clients in a client-serverarchitecture. Under one embodiment, the servers 170 can be web serversthat accept and respond to requests for a web page or other resource.However, the servers 170 or a subset thereof can also correspond todatabase, file, or application servers, among others. The computingdevice 110 and augmented reality glasses 140 can access one or more ofthe servers 170 through network 160.

The augmented reality glasses 140, in conjunction with the computingdevice 110, can be employed in various contexts to render content in thereal world outside the physical dimensions of a display device. In oneexample, the augmented reality glasses 140 can project web page contentthat is above the fold on top of the browser and content that is belowthe fold below the browser. Above the fold refers to web page contentonly visible with a scroll-up action. Similarly, below the fold concernsweb page content displayed solely after a scroll-down action. Employmentof an extended augmented reality view expands the amount of content thatcan be viewed at one time. In a second example, the augmented realityglasses 140 can interact with an online shopping browser extension andproject an animation or video of falling coins 152 when savings arefound or a promotional code used during a purchase, for instance. Here,user experience is enhanced by using the extended view 150. In a thirdexample, the augmented reality glasses 140 can interact with a browserextension that generates a virtual payment card. In this instance, animage of a payment card 154, together with pertinent information such asaccount number, expiration date, and security code, can be projected inthe extended view. In this manner, the payment information is availablefor use on different web pages and across browser windows and tabs. Thissignificantly improves user experience over confining paymentinformation to a separate window or tab. In another instance, objectscan be projected within a physical space surrounding the user, such asprojecting a couch within a living room or displaying a virtual card ona table.

FIG. 2 is a block diagram of a view extension system 200 in accordancewith an aspect of this disclosure. The view extension system 200 mayinclude a view browser extension 130 and augmented reality (AR) deviceextension 202. The browser extension 130 and the AR device extension 202can be components or sub-systems, as used herein. The browser extension130 and the AR device extension can be implemented by a processorcoupled to a memory that stores instructions that, when executed, causethe processor to perform the functionality of each extension. Thebrowser extension 130 expands the functionality of a web browserexecuting on a computing device to support an extended view. The ARdevice extension 202 expands the functionality of an augmented realitydevice such as augmented reality glasses to support extended viewfunctionality and interaction with the browser extension 130. However,the browser extension 130 can achieve the desired functionality in oneembodiment by invoking an exposed application programming interface(API) with or without the AR device extension 202. Further exampledetails regarding the browser extension 130 and the AR device extensionare provided below with respect to FIGS. 3 and 4 .

FIG. 3 is a block diagram depicting an example browser extension 130.The browser extension 130 may include activation component 302, contentcomponent 304, position component 306, deactivation component 308, andaugmented reality (AR) device interface component 310. The activationcomponent 302, content component 304, position component 306,deactivation component 308, and AR device interface component 310 can beimplemented by a processor coupled to a memory that stores instructionsthat cause the processor to perform the functionality of each componentwhen executed. Consequently, a computing device can be configured to bea special-purpose device or appliance that implements the functionalityof the view extension system 130. The browser extension 130 can beembodied as a standalone browser extension executed by a web browser orintegrated within another browser extension (e.g., virtual payment card,online shopping). Further, the functionality of the browser extension130 can be embedded within the browser as native functionality or acomputing device application or portion of an application, among otherthings.

The activation component 302 is operable to request presentation by anaugmented reality device such as smart glasses. Wireless communicationcan be established between a computing device and the augmented realitydevice in one embodiment. For instance, short-range wirelesscommunication systems such as Bluetooth Wi-Fi, ZigBee, Ultra-wideband(UWB), and infrared (IR) can be utilized. The request can then be sentthrough the wireless communication medium. In an alternate embodiment,the activation component 302 can generate a visual and computer-readablecode and display the code in a browser, such as on a web page. The codecan correspond to a one-dimensional or two-dimensional barcode that canbe captured and read by an augmented reality device. The presence of thecode can comprise a request for presentation.

The content component 304 is operable to store content at a location ona network-accessible data store or server. The content can include a webpage, text, image, video, programmatic code, or a combination thereof.The location can be an internet protocol (IP), uniform resource locator(URL), or a web address in an embodiment where the content is stored ona web server. This location or address can be provided by the activationcomponent 302 through wireless communication. Alternatively, theactivation component 302 can encode the location or address within thevisual code.

The position component 306 may be operable to specify a location withinan extended view where the content is to be displayed. For example, ifthe content corresponds to web page content above the fold, the positionwould likely be specified in a top portion of the extended view. Bycontrast, if the content corresponds to web page content below the fold,the position would be the bottom portion of the extended view. A virtualcard could be positioned in a corner while coins can rain down from thetop to indicate savings. Further, content can be anchored to a specificlocation or specified with a relative or movable position. The positioninformation, or location within an extended view, can be specifiedseparately from the content or with the content.

The deactivation component 308 is configured to terminate the extendedview or presentation of content within the extended view. Thedeactivation component 308 can communicate with an augmented realitydevice and request termination of the presentation of all or a portionof the displayed content. Alternatively, a flag can be set to indicatewhether or not to terminate the presentation. The flag can be set orunset, and an action can be associated with each state. For example, thesetting of the flag can indicate that the presentation of content shouldbe terminated, while an unset flag can denote continued presentation orvice versa. An augmented reality device can poll the flag to determinewhether to present or terminate content presentation. The presentationcan be discontinued for several reasons. First, the presentation can beterminated after a predetermined time or after playing a video a setnumber of times. Second, the presentation can be terminated by closing aweb browser or browser tab. Further, a mechanism can be provided to auser to terminate the presentation of content, among other things.

The augmented reality (AR) device interface component 310 may enablecommunication and interaction with augmented reality devices, such asglasses. In one instance, the interface component 310 can enable the useof an application programming interface (API) associated with anaugmented reality device. An augmented reality device can expose a setof operations that can be invoked from the browser extension 130 and,more particularly, augmented reality device interface component 310.Other embodiments are also possible to enable communication between abrowser and augmented reality device.

FIG. 4 is a block diagram of an example augmented reality (AR) deviceextension 202 to support view extension on an augmented reality device.The AR device extension 202 can extend or upgrade the nativefunctionality of an augmented reality device, such as augmented realityglasses, to provide for view extension functionality. The AR deviceextension 202 comprises request detection component 402, codeacquisition component 404, code execution component 406, and copycomponent 408. The components can be implemented by a processor coupledto a memory that includes instructions that, when executed by theprocessor, cause the processor to implement the functionality of eachcomponent and the AR device extension 202.

The request detection component 402 is operable to detect a request forcontent presentation in an extended view. The request can be detectedbased on direct communication over a wireless network in one instance.For example, an AR device can be communicatively paired with a computingdevice to permit interaction and data exchange. In this scenario, therequest detection component 402 can monitor and analyze communicationsfrom a computing device for a presentation request or the like.

The request detection component 402 is also operable to detect a visualcode displayed by a browser or web page as the request. For example,images of the browser display can be captured by a camera associatedwith an augmented reality device and analyzed to detect the presence orabsence of a visual code to determine whether or not a request isdetected. For instance, the request detection component 402 can detect abarcode displayed by a web browser. In addition to the request, or aspart of the request, a uniform resource locator, web address, or othernetwork location can be specified. For instance, the visual code canencode a web address.

The code acquisition component 404 is operable to acquire code or othercontent from a network location. The code acquisition component 404 canacquire a location of a network-accessible store or server from or inconjunction with the request. In one instance, provisioning a networkaddress itself or a representation of the network address corresponds toa presentation request. The code acquisition component 404 can acquireor extract the network address from a visual representation.Subsequently, the code acquisition component 404 can navigate to thenetwork location and receive code, content, or the like. The code caninclude functionality or content associated with an extended view.

The code execution component 406 can receive code acquired by the codeacquisition component 404 and trigger execution. According to oneaspect, the code can correspond to content to be projected at aparticular location associated with a browser and computing devicedisplay. Execution can correspond to invoking one or more augmentedreality device functions or procedures to cause the content to bepresented. As per another aspect, code can pertain to web page contentfor presentation above or below the browser or computing device display.In this situation, a camera can track, or a browser can identify,portions of content currently being displayed. Based on this input, codeexecution can determine the portions of content that are above and belowthe fold and initiate presenting of that content in the appropriatelocations. The code execution component 406 can also query a flag todetermine whether content presentation should continue or be terminated.The code execution component 406 enables simple and highly complexpresentation logic to be specified and performed by an augmented realitydevice.

The copy component 408 is operable to enable the copying of contentdisplayed in an augmented reality extended view. Mechanisms such asphysical buttons or gestures can be provided to enable the selection andcopying of content displayed in augmented reality. For example, a usercan gaze in the direction of content, tap once at a location on theuser's augmented reality glasses to select the content, and tap thelocation again to copy the content. The content can then be communicatedto a web browser executing a separate computing device and saved to theclipboard for subsequent pasting. Per one embodiment, anetwork-accessible clipboard can be employed such that content can besaved to it and accessed as part of a paste operation, for instance.

By way of example, and not limitation, disclosed embodiments may addressa situation in which a virtual payment card is presented in the top leftcorner of an extended view, and payment information is solicited for anonline purchase through the user's web browser. In this instance, thecomputer system may include sensors that detect a gaze of the user inthe direction of a graphic of a payment card including overlaid paymentinformation, and include hardware (e.g., a button, touchscreen) toreceive a user command to copy the payment information. For example, thecomputing system may include a sensor that detects a users' tap at alocation on the user's augmented reality glasses and use that to triggerto copy the payment information. The augmented reality device can sendthe payment information to the browser by way of wireless communicationtechnology, and the computing system operating the browser can save thepayment information to a local clipboard. Alternatively, oradditionally, the augmented reality device can save the paymentinformation to a network-accessible clipboard that can be accessed bythe computing device that displays the browser. The computing device maythen receive as input a user command to paste the information into thebrowser to satisfy the request for such information.

The aforementioned systems, architectures, platforms, environments, orthe like have been described with respect to interaction between severalcomponents. It should be appreciated that such systems and componentscan include those components or sub-components specified therein, someof the specified components or sub-components, and/or additionalcomponents. Sub-components could also be implemented as componentscommunicatively coupled to other components rather than included withinparent components. Further yet, one or more components and/orsub-components may be combined into a single component to provideaggregate functionality. Communication between systems, componentsand/or sub-components can be accomplished following either a push and/orpull control model. The components may also interact with one or moreother components not specifically described herein for the sake ofbrevity but known by those of skill in the art.

Various portions of the disclosed systems above and methods below caninclude or employ artificial intelligence, machine learning, orknowledge or rule-based components, sub-components, processes, means,methodologies, or mechanisms (e.g., support vector machines, neuralnetworks, expert systems, Bayesian belief networks, fuzzy logic, datafusion engines, classifiers). Such components, among others, canautomate certain mechanisms or processes performed, thereby makingportions of the systems and methods more adaptive as well as efficientand intelligent. By way of example, and not limitation, the viewextension system 200 can employ such mechanisms for at least imageanalysis with respect to identifying a displayed code and extracting anetwork location. Additionally, such mechanisms can be employed toprovide computer vision and analysis to detect a browser and enable theextended view to be appropriately positioned surrounding the browser.

In view of the example systems described above, methods that may beimplemented in accordance with the disclosed subject matter will bebetter appreciated with reference to flow chart diagrams of FIGS. 5-7 .While for purposes of simplicity of explanation, the methods are shownand described as a series of blocks, it is to be understood andappreciated that the disclosed subject matter is not limited by theorder of the blocks, as some blocks may occur in different orders and/orconcurrently with other blocks from what is depicted and describedherein. Moreover, not all illustrated blocks may be required toimplement the methods described hereinafter. Further, each block orcombination of blocks can be implemented by computer programinstructions that can be provided to a processor to produce a machine,such that the instructions executing on the processor create a means forimplementing functions specified by a flow chart block.

Turning attention to FIG. 5 , a method of presenting an extended view500 is illustrated. The method 500 can be implemented and executed bythe view extension system 200 and, more particularly, the browserextension 130.

At step 510, the method 500 may identify content to present ofsubstantially any type. For example, the content can include a web pageor portion thereof, an image, a video, an animation, programmatic code,or a combination. The content can be associated with a web page or otherbrowser extension (e.g., online shopping, virtual payment number . . .).

At step 520, the method 500 may determine a presentation position. Theposition pertains to the location within an extended view at which thecontent will be projected. For example, the position can be above orbelow a browser, top left corner, or bottom right corner, among others.In one instance, the position can be anchored such that content will notmove relative to the browser and display. Alternatively, the positioncan be relative and moveable with a user's gaze. For example, if thecontent is a piece of furniture, a rendering can move with a user's gazeto allow the user to picture how the piece of furniture would look andfit within a room.

At step 530, the method 500 may communicate the content and position toan augmented reality device. The content can be communicated wirelesslyto augmented reality glasses in one embodiment. For example, augmentedreality glasses (e.g., augmented reality glasses 140) can be wirelesslypaired with a computing device exposing the browser. Alternatively, codecan be displayed by a web browser on a web page that encodes the contentand position or a network location where the content and position aresaved. For example, the code can correspond to a two-dimensional barcodethat specifies a web address (e.g., IP address, URL, . . . ) thatincludes the content and position data.

At step 540, the method 500 may determine whether or not to terminatepresenting the content. In one instance, content can be presented forsolely a predetermined time. Further, content presentation can beterminated after closing a web browser or particular tab associated withthe content. Furthermore, a user can specify that the presentation ofcontent be terminated. If content presentation should not be terminated(“NO”), the method 500 can loop back to step 540 to check again. Ifcontent presentation should be terminated (“YES”), the method 500continues at step 550.

At step 550, the method 500 may send a termination request to terminatepresenting to an augmented reality device. The termination request canbe sent directly to the augmented reality device by way of establishedwireless communication. Alternatively, a flag can be set or unset at anetwork location, indicating that content presentation should beterminated. The augmented reality device can regularly or periodicallycheck the flag to determine whether to continue or terminate contentpresentation.

FIG. 6 is a flow chart diagram of an augmented reality presentationmethod 600. The method 600 can be executed by the augmented realitydevice extension 202 as part of the view extension system 200.

At step 610, the method 600 may detect a request for extending the view.The request can be detected by analyzing wireless communications for therequest. Alternatively, a camera can capture images of web browserdisplays, and the images can be analyzed for the presence of a visualcode indicative of the request. For example, the visual code can be abarcode or the like.

At step 620, the method 600 may identify a network location based on therequest. In one instance, the request can be communicated wirelessly inconjunction with a network location that can be identified. In aninstance in which a visual code is employed, the network location can beextracted from the visual code. For example, a two-dimensional barcodeor QR code can encode a network location that can be read.

At step 620, the method 600 may acquire content and position data, amongother things, from the identified network location. The network locationcan correspond to a network-accessible database or server. The contentcan correspond to a web page, image, video, animation, programmaticcode, or a combination thereof. Position data can also be specified forthe content. For example, the position data can specify a locationwithin an extended view where the content is to be displayed.

At step 630, the method 600 may render the content in an extended viewat the position. In one instance, rendering can correspond to projectingthe content on one or more lenses of augmented reality glasses. Theextended view can circumscribe a browser and display rendering browsercontent. A user can view the browser as it is displayed. However, theview surface is extended beyond the purview of the browser and displaydevice with augmented reality glasses. In this manner, invisible contentthat requires scrolling in a browser to become visible (e.g., above thefold, below the fold) can now become visible in an extended virtualreality view. Further, information conventionally confined to a browsertab can be rendered outside the tab in the extended view for use withall tabs or outside the browser. For example, virtual credit cardinformation can be rendered in augmented reality and used to enter theinformation in different tabs or outside of a computing environment.

At step 650, the method 500 may determine whether to terminate renderingthe content. Rendering can be terminated for several reasons. Forinstance, content can be designated for display for solely apredetermined time. Content can also be terminated when a browser orbrowser tab is closed, or a user requests content presentationtermination. A termination request can be sent to and received by anaugmented reality device wirelessly. Alternatively, a flag or the likecan be set or unset at a network location, which the augmented realitydevice can poll. If the rendering is to continue (“YES”), the method 600returns to step 640. If the rendering is not to continue (“NO”), themethod continues at step 660, where the augmented reality device ceasesor terminates the rendering of the content.

FIG. 7 is a flow chart diagram of a method 700 of content copying froman extended view. The method 700 can be implemented and executed as partof the view extension system 200.

At step 710, the method 700 may copy content presented in an extendedaugmented reality view. In one instance, a user can gaze at contentdisplayed in virtual reality and then perform an action to triggercopying. For example, the user can press a physical or virtual button ortap a number of times at a location on a pair of augmented realityglasses. The content can include an image or graphic or informationassociated therewith. For instance, the content can include an image ofa credit card with credit card information overlaid on top of the image.In this case, the image, the credit card information, or both can becopied.

At step 720, the method 700 may transmit copied content to a usercomputing device. The content can be sent directly to the computingdevice through established wireless communication. Alternatively, thecontent can be sent and saved to a network-accessible location.

At step 720, the method 700 may add the copied content to a computingdevice clipboard. The computing device can receive the content by way ofwireless communication. Alternatively, the content can be received orretrieved from a network-accessible data store or server. Once acquired,the content can be saved to the local clipboard.

At step 740, the method 700 may paste content from the clipboard to alocation. For example, credit card information previously presented inan extended augmented reality view and subsequently copied and saved tothe clipboard can be utilized to complete payment fields on a web page.

This disclosure pertains to the technical problem of web browsers andlimited display capabilities. More specifically, web browsers areconfined to presenting content within the physical bounds of a displaydevice. The technical solution comprises extending the view by way ofaugmented reality glasses. A web browser, through a browser extension,can communicate a request to present content in an extended augmentedreality view. As part of or separate from the request, content andpositioning information, among other things, can be provided orotherwise made available to augmented reality glasses. After receipt,the augmented reality glasses can project the content on one or morelenses above, below, or on the sides of a browser or display device. Auser can subsequently view the browser content and content presented inthe extended view.

Aspects of the subject disclosure have been described extensively withrespect to a web browser. However, the aspects are not limited toemployment with respect to a browser. Instead, the aspects can beutilized with substantially any application on any device. For example,display of an application executing on a smartphone or watch can beextended beyond the physical dimensions of those devices with anaugmented reality device.

Further, a display can be extended by an augmented reality device beyonda predetermined distance surrounding a display device. In one instance,the display can be extended to a predetermined distance surrounding theuser. A user's current position can be determined or inferred based onvarious technologies alone or in combination, including global satellitepositioning (GPS), wireless location beacons or tags, andcomputer-vision-based object detection. In this manner, content can berendered in a physical location surrounding a user, such as in theliving room. For example, a safe could be rendered in the living room orbags of money displayed on a couch in the living room.

As used herein, the terms “component” and “system,” as well as variousforms thereof (e.g., components, systems, sub-systems), are intended torefer to a computer-related entity, either hardware, a combination ofhardware and software, software, or software in execution. For example,a component may be but is not limited to being a process running on aprocessor, a processor, an object, an instance, an executable, a threadof execution, a program, and/or a computer. By way of illustration, bothan application running on a computer and the computer can be acomponent. One or more components may reside within a process and/orthread of execution, and a component may be localized on one computerand/or distributed between two or more computers.

As used herein, the terms “infer” and “inference” generally refer to theprocess of reasoning about or inferring states of a system, a component,an environment, or a user from one or more observations captured by wayof events or data, among other things. Inference may be employed toidentify a context or an action or may be used to generate a probabilitydistribution over states, for example. An inference may beprobabilistic. For example, computation of a probability distributionover states of interest can be based on a consideration of data orevents. Inference may also refer to techniques employed for composinghigher-level events from a set of events or data. Such inference mayresult in the construction of new events or new actions from a set ofobserved events or stored event data, whether or not the events arecorrelated in close temporal proximity, and whether the events and datacome from one or several events and data sources.

The conjunction “or” as used in this description and appended claims isintended to mean an inclusive “or” rather than an exclusive “or,” unlessotherwise specified or clear from the context. In other words, “‘X’ or‘Y’” is intended to mean any inclusive permutations of “X” and “Y.” Forexample, if “‘A’ employs ‘X,’” “‘A employs ‘Y,’” or “‘A’ employs both‘X’ and ‘Y,’” then “‘A’ employs ‘X’ or ‘Y’” is satisfied under any ofthe preceding instances.

Furthermore, to the extent that the terms “includes,” “contains,” “has,”“having,” or variations in form thereof are used in either the detaileddescription or the claims, such terms are intended to be inclusive in amanner similar to the term “comprising” as “comprising” is interpretedwhen employed as a transitional word in a claim.

To provide a context for the disclosed subject matter, FIG. 8 , as wellas the following discussion, are intended to provide a brief, generaldescription of a suitable environment in which various aspects of thedisclosed subject matter can be implemented. However, the suitableenvironment is solely an example and is not intended to suggest anylimitation on the scope of use or functionality.

While the above-disclosed system and methods can be described in thegeneral context of computer-executable instructions of a program thatruns on one or more computers, those skilled in the art will recognizethat aspects can also be implemented in combination with other programmodules or the like. Generally, program modules include routines,programs, components, data structures, among other things, which performparticular tasks and/or implement particular abstract data types.Moreover, those skilled in the art will appreciate that the abovesystems and methods can be practiced with various computer systemconfigurations, including single-processor, multi-processor ormulti-core processor computer systems, mini-computing devices, servercomputers, as well as personal computers, hand-held computing devices(e.g., personal digital assistant (PDA), smartphone, tablet, watch),microprocessor-based or programmable consumer or industrial electronics,and the like. Aspects can also be practiced in distributed computingenvironments where tasks are performed by remote processing deviceslinked through a communications network. However, some, if not allaspects, of the disclosed subject matter can be practiced on standalonecomputers. In a distributed computing environment, program modules maybe located in one or both of local and remote memory devices.

With reference to FIG. 8 , illustrated is an example computing device800 (e.g., desktop, laptop, tablet, watch, server, hand-held,programmable consumer or industrial electronics, set-top box, gamesystem, compute node). The computing device 800 includes one or moreprocessor(s) 810, memory 820, system bus 830, storage device(s) 840,input device(s) 850, output device(s) 860, and communicationsconnection(s) 870. The system bus 830 communicatively couples at leastthe above system constituents. However, the computing device 800, in itssimplest form, can include one or more processors 810 coupled to memory820, wherein the one or more processors 810 execute variouscomputer-executable actions, instructions, and or components stored inthe memory 820.

The processor(s) 810 can be implemented with a general-purposeprocessor, a digital signal processor (DSP), an application-specificintegrated circuit (ASIC), a field-programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A general-purpose processor maybe a microprocessor, but in the alternative, the processor may be anyprocessor, controller, microcontroller, or state machine. Theprocessor(s) 810 may also be implemented as a combination of computingdevices, for example, a combination of a DSP and a microprocessor, aplurality of microprocessors, multi-core processors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration. In one embodiment, the processor(s) 810 can be a graphicsprocessor unit (GPU) that performs calculations concerning digital imageprocessing and computer graphics.

The computing device 800 can include or otherwise interact with avariety of computer-readable media to facilitate control of thecomputing device to implement one or more aspects of the disclosedsubject matter. The computer-readable media can be any available mediaaccessible to the computing device 800 and includes volatile andnon-volatile media, and removable and non-removable media.Computer-readable media can comprise two distinct and mutually exclusivetypes: storage media and communication media.

Storage media includes volatile and non-volatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer-readable instructions, data structures,program modules, or other data. Storage media includes storage devicessuch as memory devices (e.g., random access memory (RAM), read-onlymemory (ROM), electrically erasable programmable read-only memory(EEPROM), magnetic storage devices (e.g., hard disk, floppy disk,cassettes, tape), optical disks (e.g., compact disk (CD), digitalversatile disk (DVD)), and solid-state devices (e.g., solid-state drive(SSD), flash memory drive (e.g., card, stick, key drive)), or any otherlike mediums that store, as opposed to transmit or communicate, thedesired information accessible by the computing device 800. Accordingly,storage media excludes modulated data signals as well as that which isdescribed with respect to communication media.

Communication media embodies computer-readable instructions, datastructures, program modules, or other data in a modulated data signalsuch as a carrier wave or other transport mechanism and includes anyinformation delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, radio frequency (RF), infrared, and other wireless media.

The memory 820 and storage device(s) 840 are examples ofcomputer-readable storage media. Depending on the configuration and typeof computing device, the memory 820 may be volatile (e.g., random accessmemory (RAM)), non-volatile (e.g., read-only memory (ROM), flashmemory), or some combination of the two. By way of example, the basicinput/output system (BIOS), including basic routines to transferinformation between elements within the computing device 800, such asduring start-up, can be stored in non-volatile memory, while volatilememory can act as external cache memory to facilitate processing by theprocessor(s) 810, among other things.

The storage device(s) 840 include removable/non-removable,volatile/non-volatile storage media for storage of vast amounts of datarelative to the memory 820. For example, storage device(s) 840 include,but are not limited to, one or more devices such as a magnetic oroptical disk drive, floppy disk drive, flash memory, solid-state drive,or memory stick.

Memory 820 and storage device(s) 840 can include or have stored thereinoperating system 880, one or more applications 886, one or more programmodules 884, and data 882. The operating system 880 acts to control andallocate resources of the computing device 800. Applications 886 includeone or both of system and application software and can exploitmanagement of resources by the operating system 880 through programmodules 884 and data 882 stored in the memory 820 and/or storagedevice(s) 840 to perform one or more actions. Accordingly, applications886 can turn a general-purpose computer 800 into a specialized machinein accordance with the logic provided thereby.

All or portions of the disclosed subject matter can be implemented usingstandard programming and/or engineering techniques to produce software,firmware, hardware, or any combination thereof to control the computingdevice 800 to realize the disclosed functionality. By way of example andnot limitation, all or portions of the view extension system 200 can be,or form part of, the application 886, and include one or more modules884 and data 882 stored in memory and/or storage device(s) 840 whosefunctionality can be realized when executed by one or more processor(s)810.

In accordance with one particular embodiment, the processor(s) 810 cancorrespond to a system on a chip (SOC) or like architecture including,or in other words integrating, both hardware and software on a singleintegrated circuit substrate. Here, the processor(s) 810 can include oneor more processors as well as memory at least similar to theprocessor(s) 810 and memory 820, among other things. Conventionalprocessors include a minimal amount of hardware and software and relyextensively on external hardware and software. By contrast, a SOCimplementation of a processor is more powerful, as it embeds hardwareand software therein that enable particular functionality with minimalor no reliance on external hardware and software. For example, the viewextension system 200 and/or functionality associated therewith can beembedded within hardware in a SOC architecture.

The input device(s) 850 and output device(s) 860 can be communicativelycoupled to the computing device 800. By way of example, the inputdevice(s) 850 can include a pointing device (e.g., mouse, trackball,stylus, pen, touchpad), keyboard, joystick, microphone, voice userinterface system, camera, motion sensor, and a global positioningsatellite (GPS) receiver and transmitter, among other things. The outputdevice(s) 860, by way of example, can correspond to a display device(e.g., liquid crystal display (LCD), light emitting diode (LED), plasma,organic light-emitting diode display (OLED)), speakers, voice userinterface system, printer, and vibration motor, among other things. Theinput device(s) 850 and output device(s) 860 can be connected to thecomputing device 800 by way of wired connection (e.g., bus), wirelessconnection (e.g., Wi-Fi, Bluetooth), or a combination thereof

The computing device 800 can also include communication connection(s)870 to enable communication with at least a second computing device 802utilizing a network 890. The communication connection(s) 870 can includewired or wireless communication mechanisms to support networkcommunication. The network 890 can correspond to a personal area network(PAN), local area network (LAN), or a wide area network (WAN) such asthe Internet. In one instance, the computing device 800 can correspondto a user's desktop, laptop, or tablet computer with a web browser thatis extended by a browser extension provided by the view extension system200. The second computing device can correspond to augmented realityglasses. The computing device 800 can communicate content over thenetwork 890 for presentation by the augmented reality glasses in anextended view associated with the browser. In another instance, thecomputing device 800 can correspond to a user computing device oraugmented reality device, and the second computing device can be a datastore or server accessible over the network 890.

What has been described above includes examples of aspects of theclaimed subject matter. It is, of course, not possible to describe everyconceivable combination of components or methods to describe the claimedsubject matter. However, one may recognize that many furthercombinations and permutations of the disclosed subject matter arepossible. Accordingly, the disclosed subject matter is intended toembrace all such alterations, modifications, and variations that fallwithin the spirit and scope of the appended claims.

What is claimed is:
 1. A system for extending a view of web content withaugmented reality, comprising: one or more processors; one or morenon-transitory computer-readable media having instructions that, whenexecuted by the one or more processors, cause the one or more processorsto: identify content associated with a webpage being displayed on a webbrowser for presentation outside a purview of the web browser;determine, based on the webpage being displayed on the web browser,whether the content corresponds to web page content above a fold on topof the web browser or web page content below the fold below the webbrowser, wherein web page content above the fold includes web pagecontent only visible with a scroll-up action, wherein web page contentbelow the fold includes web page content only visible with a scroll-downaction; in response to determining that the content corresponds to webpage content above the fold, determine position information for thecontent in a top portion of an extended view overlaid on the webbrowser; in response to determining that the content corresponds to webpage content below the fold, determine the position information for thecontent in a bottom portion of an extended view overlaid on the webbrowser; generate an instruction to save the content tonetwork-accessible storage; and generate a request, including a locationof the network-accessible storage and the position information for thecontent, to an augmented reality device to present the content in theextended view overlaid on the web browser.
 2. The system of claim 1,wherein the instructions further cause the one or more processors toidentify content associated with a web browser extension that generatesa virtual payment card.
 3. The system of claim 1, wherein theinstructions further cause the one or more processors to identifycontent related to a web browser extension that identifies a deal on aproduct or service during online shopping.
 4. The system of claim 1,wherein the instructions further cause the one or more processors togenerate a visible code and display the visible code on a webpage of theweb browser to send the request.
 5. The system of claim 1, wherein theinstructions further cause the one or more processors to initiatewireless communication with the augmented reality device.
 6. The systemof claim 1, wherein the instructions further cause the one or moreprocessors to set a flag to continue presentation of the content in theextended view.
 7. The system of claim 1, wherein the instructionsfurther the one or more processors to set a flag to discontinuepresentation of the content in the extended view.
 8. The system of claim1, wherein the instructions further the one or more processors to:detect closing of the web browser; and set a flag to discontinuepresentation of the content in the extended view.
 9. The system of claim1, wherein the instructions further cause the one or more processors to:acquire data copied from the extended view; and save the data to aclipboard to enable the data to be pasted on a web page in the webbrowser.
 10. The system of claim 1, wherein the instructions furthercause the one or more processors to save a display position of thecontent in the extended view at the location.
 11. The system of claim 8,wherein the content includes persistent content, wherein discontinuingpresentation of the content in the extended view excludes discontinuingpresentation of the persistent content.
 12. The system of claim 4,wherein generating the visible code comprising generating a quickresponse code (QR code), wherein the QR code corresponds to a locationto retrieve the content for presentation in the extended view.
 13. Amethod of extending a view of web content, comprising: identifyingcontent associated with a webpage being displayed on a web browser forpresentation outside a purview of a web browser; determining, based onthe webpage being displayed on the web browser, whether the contentcorresponds to web page content above a fold on top of the web browseror web page content below the fold below the web browser, wherein webpage content above the fold includes web page content only visible witha scroll-up action, wherein web page content below the fold includes webpage content only visible with a scroll-down action; in response todetermining that the content corresponds to web page content above thefold, determining position information for the content in a top portionof an extended view overlaid on the web browser; in response todetermining that the content corresponds to web page content below thefold, determining the position information for the content in a bottomportion of an extended view overlaid on the web browser; generating aninstruction to save the content to a network-accessible data store; andgenerating a request, including a location of the network-accessibledata store and the position information for the content, to an augmentedreality device to present the content in the extended view overlaid onthe web browser.
 14. The method of claim 13, further comprisingidentifying non-displayed web page content for presentation outside thepurview of the web browser.
 15. The method of claim 13, furthercomprising identifying web page content generated by a web browserextension that generates a virtual payment card.
 16. The method of claim13, further comprising identifying web page content generated by anonline shopping browser extension.
 17. The method of claim 13, furthercomprising setting a flag to discontinue presentation of the content inthe extended view.
 18. The method of claim 13, further comprisinggenerating and displaying a visible code on a web page of the webbrowser to send the request.